Henan Province
Conformal Prediction Assessment: A Framework for Conditional Coverage Evaluation and Selection
Zhou, Zheng, Zhang, Xiangfei, Tao, Chongguang, Yang, Yuhong
Conformal prediction provides rigorous distribution-free finite-sample guarantees for marginal coverage under the assumption of exchangeability, but may exhibit systematic undercoverage or overcoverage for specific subpopulations. Assessing conditional validity is challenging, as standard stratification methods suffer from the curse of dimensionality. We propose Conformal Prediction Assessment (CPA), a framework that reframes the evaluation of conditional coverage as a supervised learning task by training a reliability estimator that predicts instance-level coverage probabilities. Building on this estimator, we introduce the Conditional Validity Index (CVI), which decomposes reliability into safety (undercoverage risk) and efficiency (overcoverage cost). We establish convergence rates for the reliability estimator and prove the consistency of CVI-based model selection. Extensive experiments on synthetic and real-world datasets demonstrate that CPA effectively diagnoses local failure modes and that CC-Select, our CVI-based model selection algorithm, consistently identifies predictors with superior conditional coverage performance.
- Oceania > New Zealand (0.04)
- Europe > Germany (0.04)
- Europe > Finland > Northern Ostrobothnia > Oulu (0.04)
- (4 more...)
- Semiconductors & Electronics (0.64)
- Information Technology (0.47)
- Transportation (0.46)
- North America > United States > Minnesota > Hennepin County > Minneapolis (0.28)
- North America > Dominican Republic (0.04)
- Europe > Ireland > Leinster > County Dublin > Dublin (0.04)
- (18 more...)
- Health & Medicine > Therapeutic Area (1.00)
- Government (0.93)
- Information Technology > Artificial Intelligence > Representation & Reasoning (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Chatbot (0.95)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.70)
- Europe > Switzerland > Zürich > Zürich (0.14)
- North America > Canada > Quebec > Montreal (0.05)
- North America > United States > Louisiana > Orleans Parish > New Orleans (0.05)
- (9 more...)
- Information Technology > Sensing and Signal Processing > Image Processing (1.00)
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks (0.68)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (0.68)
- Asia > Myanmar > Tanintharyi Region > Dawei (0.04)
- Asia > China > Henan Province > Zhengzhou (0.04)
- South America > Chile > Santiago Metropolitan Region > Santiago Province > Santiago (0.04)
- Asia > China > Tianjin Province > Tianjin (0.04)
- Information Technology > Security & Privacy (0.93)
- Law (0.68)
- Asia > China > Hong Kong (0.05)
- Asia > Myanmar > Tanintharyi Region > Dawei (0.04)
- Asia > China > Tianjin Province > Tianjin (0.04)
- (6 more...)
- Asia > China > Hong Kong (0.04)
- Africa > Ethiopia > Addis Ababa > Addis Ababa (0.04)
- North America > United States > California > San Diego County > San Diego (0.04)
- (8 more...)